Not Knowing Your Random Number Generator Could Be Costly

Introduction Have you played a video game lately? Purchased something on the web? Trained a Neural Network? Used a Genetic Algorithm for optimization? Run software from Microsoft? Applied textures to a photograph? Played the stock market? If the answer is “yes” to any of these questions, Random Number Generator (RNG) have affected your life. Can you trust your Random Number Generator? Can you test its performance? Does its performance really matter? In addition to some RNG related horror stories, this article presents guidelines for performing basic random number evaluations. It also provides numerous useful links and resources. My interest in random numbers was re-kindled recently while determining whether a particular neural network prediction was better than chance or any of several other prediction algorithms. I was using a Monte-Carlo simulation in Microsoft Visual Basic (tied into Microsoft Excel). To ensure that the code was working correctly, I selected several outcomes for which I could compute the probability of occurrences and associated variance. With 1,000,000 trials, I expected three hits, yet I did not receive one. Even after repeating the experiment several more times, I never obtained the expected outcome, which was highly improbable. Out of frustration, I modified the output of the Visual Basic random number generator by shuffling the numbers using a Bays-Durham Shuffle (a method for shuffling the order of the output of a random number generator as shown in figure 1). This reduced the correlation between successively generated numbers that interacted with my code, and I started approaching the expected number of hits (0-5 hits per run of 1,000,000,000 trials). Later testing verified that the Visual Basic RNG fails several common tests for random numbers. However, as discussed later, when the Visual Basic RNG is used in conjunction with a BaysDurham shuffle, it passes more of the tests. This experience led me to start collecting RNGs, and methods for testing them. I am currently developing a software package that provides access to over 250 commonly used RNGs, and 13,000 lesser-known RNGs and variants. This software will be available shortly at www.extremeet.com.